1. Field of the Invention
The present invention relates to an image processing device for compressing the dynamic range of an input image, an image processing method for compressing the dynamic range of an input image, and an image pickup apparatus incorporated with the image processing device.
2. Description of the Related Art
In recent years, in an image pickup apparatus adapted as a digital camera, such as a digital still camera or a digital video camera, as a demand for high-quality performance is increased, there is a demand of increasing the luminance range i.e. the dynamic range of a subject image. Various techniques for increasing the dynamic range have been studied and developed. The luminance range corresponds to a difference between a lowest luminance and a highest luminance, and in the case of an image, the luminance range corresponds to a difference between a lowest density and a highest density.
Japanese Unexamined Patent Publication No. Sho 63-306779 (D1) discloses a technique, for increasing the dynamic range, comprising: shooting multiple images having different exposure amounts, selecting image portions having a proper exposure level from the multiple images, and combining the selected image portions. Japanese Unexamined patent Publication No. 2000-165754 (D2) discloses a technique, for increasing the dynamic range, comprising: storing signal charges accumulated in a photoelectric converter by a one-time exposure in multiple capacitances, reading out the signal charges stored in the capacitances having different capacities, and summing up the readout signal charges. Japanese Unexamined Patent Publication No. 2000-165755 (D3) discloses a technique, for increasing the dynamic range, comprising: converting a signal charge transferred from a photoelectric converter into a signal voltage by a charge-voltage converter constituted of multiple capacitances having different voltage dependencies.
U.S. Pat. No. 6,927,884 corresponding to Japanese Unexamined Patent Publication No. 2002-77733 (D4) discloses a solid-state image sensing device, for increasing the dynamic range, comprising: a photosensitive member operable to generate a photocurrent in accordance with an incident light amount; an MOS transistor for receiving the photocurrent; and a biasing member for biasing the MOS transistor to such a state that a sub-threshold current is allowed to flow, wherein the photocurrent is logarithmically converted into an electric signal in accordance with the incident light amount for outputting the electric signal. D4 proposes a solid-state image sensor for converting a photocurrent into a logarithmic voltage by using a sub-threshold characteristic of the MOS transistor for increasing the dynamic range, wherein an output characteristic inherent to the solid-state image sensor is automatically switched between a linear condition where a photocurrent is linearly converted into an electric signal in accordance with the incident light amount for outputting the electric signal, and the aforementioned logarithmic condition, by applying a specific reset voltage to the MOS transistor.
Under the aforementioned circumstances that the technique for increasing the dynamic range has progressed in the field of image pickup apparatuses, in a current technical standard, however, there is a case that the dynamic range (i.e. the bit number for expressing the pixel level) in a device for transferring, accumulating, or displaying an image is relatively narrow, as compared with the dynamic range in the image pickup apparatus. In the above occasion, even if the dynamic range in the image pickup apparatus is successfully increased, it may be difficult to utilize the entirety of the obtained information. Accordingly, a dynamic range compression technique for converting an image having a wide dynamic range into an image having a smaller dynamic range has also been studied and developed.
A number of dynamic range compression techniques based on the Retinex theory (E. H. Land, J. J. McCann, “Lightness and retinex theory”, Journal of the Optical Society of America 61(1), 1 (1971)) have been reported. According to the Retinex theory, whereas light to be incident to a human eye is determined by a product of illumination light and a reflectance of an object, visual sensation of the human eye has a strong correlation to the reflectance. In view of the Retinex theory, in an image having a wide dynamic range, a reflectance component having a strong correlation to visual sensation of the human eye can be maintained by exclusively reducing the dynamic range of the illumination light component. Thereby, an image having a high contrast and a compressed dynamic range can be obtained. In other words, an image having a compressed dynamic range can be obtained by adjusting the density of a bright portion and a dark portion, while maintaining the gradation of an image portion having an intermediate density.
U.S. Pat. No. 6,807,316, corresponding to Japanese Unexamined Patent Publication No. 2001-298619 (D5) discloses a technique, for compressing the dynamic range of an original image, comprising: generating a low frequency component of an image signal S0 i.e. creating an unsharp image S1 having a moderate change in luminance by subjecting an image signal S0 representing an original image to low-pass filter processing; inverting the value of the unsharp image signal S1 by performing data conversion with respect to the unsharp image signal S1 based on a lookup table; creating a processed image signal S4 whose dynamic range is compressed; and obtaining an image signal S5 by summing up the processed image signal S4 and the image signal S0 representing the original image. In the dynamic range compression technique disclosed in D5, in the case where the dynamic range is unduly compressed, a halo effect may be generated, wherein a pseudo outline in the form of a band having a certain width is generated along a boundary e.g. the outline of a subject image, where the luminance is sharply changed, as shown in a boundary between the subject image and a background image. In view of this, D5 proposes an image processing method, capable of suppressing generation of a pseudo outline, comprising: creating multiple unsharp image signals representing unsharp images of an original image, based on an image signal representing the original image; generating one combined unsharp image signal by combining the multiple unsharp image signals; and subjecting the image signal representing the original image to dynamic range compression based on the combined unsharp image signal. A compression function ftotal (α) to be used in the dynamic range compression in the image processing method proposed in D5 is generated by: calculating a total compression ratio α having a profile shown in FIG. 12 to define a compression function f(α); and correcting the compression function f(α) with a compression function flight (α) with respect to a bright portion and a compression function fdark(α) with respect to a dark portion. The total compression ratio α having the profile shown in FIG. 12 is set to 0, in the case where the dynamic range is smaller than a predetermined threshold value DRth; fixed to a lower limit value αmax, in the case where the dynamic range is larger than the threshold value DRmax (>DRth); and is linearly changed in the case where the dynamic range is not smaller than the threshold value DRth and not larger than the threshold value DRmax.
Japanese Patent No. 3,750,797 (D6) discloses an image processing method, for converting an input image into an image having a relatively small dynamic range, comprising: a smoothing step of performing smoothing processing with respect to each of divided input images to generate smoothed images having different degrees of smoothness; an edge intensity calculating step of calculating an edge intensity based on the smoothed images; a combining step of combining the smoothed images based on the calculated edge intensity; a coefficient calculating step of calculating a coefficient to be used in converting each pixel value of the input images, based on a combined smoothed image generated by combining the multiple smoothed images; and a pixel value converting step of converting each pixel value of the input images based on the calculated coefficients. In the dynamic range compression processing to be used in the image processing method disclosed in D6, dynamic range compression is performed by: calculating a coefficient C(x,y)=(F(R(x,y))), using a coefficient calculation function F(1) having a profile shown in FIG. 13A, based on each pixel value R(x,y) of the combined smoothed image R; and multiplying a pixel value I(x,y) of the input image I by the coefficient C(x,y). The coefficient calculation function F(1) is calculated by performing a computation: F(1)=T(l)/1, using a level conversion function T(l) having a profile shown in FIG. 13B. A minimum value Cmin of the coefficient C is given by a ratio Mmax/Lmax, which is a ratio of the maximum value Mmax of output level to the maximum value Lmax of input level in the level conversion function T(l). Alternatively, a gamma function T(l)=(1/Lmax)g×Lmax, a LOG function T(l)=(log(1)/log(Lmax))×Lmax, or a histogram equalization method, wherein the level conversion function is adaptively changed depending on a frequency distribution of the pixel level of an input image, may be used, as the level conversion function T(l).
In the dynamic range compression techniques disclosed in D5 and D6, the image quality may be degraded depending on an input image in view of a point that the aforementioned function is used. For instance, in D6, the minimum value Cmin of the coefficient C is determined by the maximum value (Mmax, Lmax) of the level conversion function T(l). Accordingly, in the case where an input image includes one or more pixels having an exceedingly large pixel value, most of the pixels in the input image are outputted as a dark image without compression, which may degrade the image quality.